Method for assuring that self - imposed changes made by adaptive systems do not compromise safety

This article presents a method consisting of 5 steps for assuring that self-imposed changes made by adaptive systems do not compromise safety. The method is intended to be used within the field of safety critical systems development with respect to applications where adaptive neural networks are part of some process control loop. The method is to provide a means for assuring before commissioning that all self-imposed changes resulting from the adaptivity after commissioning will have no adverse effects. This is achieved by: 1) identifying and specifying the potential hazards induced by the adaptiveness which must be handled during operation; 2) proposing a mechanism for separating the process of control and the process of adaptation such that only self-imposed changes with no effect on safety are effectuated; 3) providing a means for assessing online that self-imposed adaptive changes do not compromise safety. no change during operation is effectuated before it has been assessed that there are no adverse effects. In order to provide this assurance, an online but out-of-the-loop adaptation and verification strategy is proposed as a mechanism to allow separation between the process of adapting the controller and the process of control; this is motivated and discussed in section 2. The strategy provides a means for safe adaptive control by allowing refined versions of the control software replace the controller operating in-theloop in an iterative adapt-verify-replace process. In order to be practically feasible, this approach requires that the processes related to the online handling of adaptation and verification can be effectively automated. This article is primarily focused on verification issues, and then on the ability to verify that the control software satisfies safety requirements. In order to establish an effective verification process, our method emphasises hazard identification and a subsequent handling supported by formal methods in the following major steps: 1. Hazard identification and analysis 2. Risk handling